Method and apparatus for standardizing ultrasonography training using image to physical space registration of tomographic volumes from tracked ultrasound

ABSTRACT

A clinical and training apparatus that collects and processes physical space data while performing an image-guided procedure on an anatomical area of interest includes a calibration probe, a tracked ultrasonic probe, a wireless tracking device that tracks the ultrasonic probe in space and an image data processor. The physical space data provides three-dimensional coordinates for each of the physical points. The image data processor includes a memory holding instructions. The instructions include determining registrations used to indicate position in image space and physical space; using the registrations to map into image space, image data describing the physical space of the tracked ultrasonic probe and the anatomical area of interest; and constructing a three-dimensional (3D) volume based on ultrasonic image data. The apparatus includes a storage medium that stores a plurality of 3D volumes acquired by the image data processor for later retrieval.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 60/720,813 filed on Sep. 27, 2005 entitled “Method and Apparatus forStandardizing Ultrasonography Training Using Image to Physical SpaceRegistration of Tomographic Volumes from Tracked Ultrasound.”

BACKGROUND OF THE INVENTION

The present invention relates to a method and apparatus forstandardizing ultrasonography training, and more particularly, a methodand apparatus for standardizing ultrasonography training using image tophysical space registration of tomographic volumes from trackedultrasound.

There are a number of medical imaging and instrumentation systemspresently available including Positron Emission Tomography (PET),Magnetic Resonance Imaging (MRI), computed tomography (CT) andultrasound. CT and MRI are imaging modalities that provide tomographicslices of the imaged volume.

CT and/or MRI used in combination with other imaging modalities, providemore useful information for the staging and treatment of cancer. Forexample, using CT and ultrasound can provide better diagnostics as wellas the potential for spatial orientation of the patient and/or the areaof interest.

A fundamental force in the development of surgery and other forms ofdirected therapy is the need to increase the information available tophysicians and to place that information in both spatial and temporalcontexts. The field of interactive image guided procedures (IGP), as itis known today, began in the 1980's and focused on tracking the surgicalposition in the physical space and display position in image space. Thistechnique was first used in the field of neurosurgery and eventuallycrossed over into many other medical fields. IGPs have four basiccomponents including image acquisition, image-to-physical-spaceregistration, three-dimensional tracking, and display of imaging dataand location. There has been much concurrent advancement in these fourareas, which is a necessity for the timely incorporation of IGP intocommon medical practice.

A system of interest utilizes a scan technology such as CT incombination with an ultrasound instrument, such as an endorectalultrasound (ERUS) probe, that is tracked for position in physical spacein order to display position in image space. Such a system provides muchmore diagnostic information, but training students and technicians oneven general ultrasonography is a somewhat complicated undertaking.Ultrasound image interpretation is associated with a significantlearning curve.

Currently practical, as opposed to didactic, ultrasonography training isperformed by trainees practicing on live patients and then learningdisease processes from mentors. The training is serendipitous at bestsince a trainee practices only on the available test patients and not awide variety of test patients suffering from a wide variety of diseasesor disorders. The current approach to training in ultrasonographygenerally utilizes classroom training and education in the theory andphysics of ultrasound measurements. Once this is completed the traineegoes through extended on the job training by practicing and learning onreal patients under the guidance of an experienced radiologist. In somespecialties this may require working with 50 to 100 patients. In aclinical setting, the flow of patients is irregular and unpredictable.Therefore, the duration of training is unpredictable. By using the timeof a radiologist or other skilled imaging specialist along with thetrainee, the time and cost expended is significant by the time thetrainee is sufficiently “certified” to independently acquire and/orinterpret images.

Thus, presently available ultrasonography training is provided withmodels that may not be realistic and certainly are limited by the numberand expense of multiple models with which to train. There are more than25,000 registered ultrasound technicians. There are differentspecialties within ultrasonography. The testing is standardized andinvolves static exams after a training and schooling period. Thus, it isdesirable to provide an ultrasonography training system that permits thetrainee to view more images of particular areas of interest forspecialization and for improving training.

One ultrasound training system is disclosed in U.S. Pat. No. 5,609,485(Bergman et al.). Bergman et al. disclose a medical “reproductionsystem” for use in training. The medical reproduction system is acomputer-based, interactive system for use by physicians and techniciansin medical training and diagnosis by means of medical instrumentationsuch as ultrasound machines. The system of Bergman et al. collectsultrasound data and creates a training system using simulated tools(e.g., ultrasound machine, transducer and mannequin). Bergman et al.suggest that the system can be used for offline diagnosis by atechnician or physician at some time after an actual patient scan orultrasound. However, Bergman et al. does not provide an assessment ofthe accuracy of the acquired ultrasound volumes, and therefore, there isno detail as to how the system would be useful as a clinical diagnostictool.

It is desirable to provide a method and apparatus for standardizingultrasonography training using image to physical space registration oftomographic volumes from tracked ultrasound. It is desirable to providea technology that would shorten, standardize, and broaden the trainingfor technicians as well as radiologists and surgeons. It is desirable touse “spatially-oriented” ultrasound images for training of physicians,technicians and nurses in the use and interpretation of ultrasoundimages for various portions of the anatomy.

BRIEF SUMMARY OF THE INVENTION

Briefly stated, the present invention comprises an apparatus and methodfor calibration, tracking and volume construction data for use inimage-guided procedures. In one embodiment, the present inventioncomprises an apparatus that collects and processes physical space datawhile performing an image-guided procedure on an anatomical area ofinterest. The apparatus includes a calibration probe that collectsphysical space data by probing a plurality of physical points, a trackedultrasonic probe that outputs two-dimensional ultrasonic image data, atracking device that tracks the ultrasonic probe in space and an imagedata processor comprising a computer-readable medium. The physical spacedata provides three-dimensional (3D) coordinates for each of thephysical points. The computer-readable medium holds computer-executableinstructions that includes determining registrations used to indicateposition in both image space and physical space based on the physicalspace data collected by the calibration probe; using the registrationsto map into image space, image data describing the physical space of thetracked ultrasonic probe used to perform the image-guided procedure andthe anatomical area of interest; and constructing a three-dimensionalvolume based on the two-dimensional ultrasonic image data on a periodicbasis.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofthe invention, will be better understood when read in conjunction withthe appended drawings. For the purpose of illustrating the invention,there are shown in the drawings embodiments which are presentlypreferred. It should be understood, however, that the invention is notlimited to the precise arrangements and instrumentalities shown. In thedrawings:

FIG. 1 shows a schematic block diagram of a hardware system for onepossible configuration of an image-guided procedure tracking system inaccordance with preferred embodiments of the present invention;

FIG. 2 is a perspective photographic view of a tracked endorectalultrasonic (TERUS) probe for use with preferred embodiments of thepresent invention;

FIG. 3 is a perspective photographic view of the tracked endorectalultrasonic (TERUS) probe of FIG. 2 being immersed in a water-bath forcalibration;

FIG. 4 is a perspective photographic view of a calibration probe and areference emitter for an optically tracked system, each having aplurality of infrared emitting diodes (IREDs) disposed thereon for usewith the image-guided procedure tracking system of FIG. 1;

FIG. 5 is a perspective photographic view of an image-guided proceduretracking system in accordance with preferred embodiments of the presentinvention;

FIG. 6 a perspective view of a fiducial marker for use with theimage-guided procedure tracking system of FIG. 1;

FIG. 7 shows a schematic block diagram of a hardware system for onepossible configuration of an optical tracking system in accordance withpreferred embodiments of the present invention;

FIG. 8 shows a general flow chart for an image-guided tracking system inaccordance with preferred embodiments of the present invention;

FIG. 9 shows a basic software architecture for one possibleconfiguration of an image-guided tracking system in accordance withpreferred embodiments of the present invention;

FIG. 10 shows a general flow chart for an image-guided tracking systemfor one possible configuration of an image-guided tracking system inaccordance with preferred embodiments of the present invention;

FIG. 11 shows a general flow chart for developing a training anddiagnostics library in accordance with preferred embodiments of thepresent invention;

FIG. 12 shows a general flow chart for training using the training anddiagnostics library of FIG. 11;

FIG. 13 is an ultrasonic image showing a tip of a calibration probeacquired by a tracked endorectal ultrasonic (TERUS) probe during acalibration procedure in accordance with the preferred embodiments ofthe present invention; and

FIG. 14 a computed tomography (CT) image of a rectal phantom having afiducial marker affixed thereto.

DETAILED DESCRIPTION OF THE INVENTION

Certain terminology is used in the following description for convenienceonly and is not limiting. The words “right”, “left”, “lower”, and“upper” designate directions in the drawings to which reference is made.The words “inwardly” and “outwardly” refer direction toward and awayfrom, respectively, the geometric center of the object discussed anddesignated parts thereof. The terminology includes the words abovespecifically mentioned, derivatives thereof and words of similar import.Additionally, the word “a”, as used in the claims and in thecorresponding portions of the specification, means “one” or “at leastone.”

Preferred embodiments of the present invention include Image GuidedProcedures (IGP). IGP have four basic components: image acquisition,image-to-physical-space registration, three-dimensional tracking, anddisplay of imaging data and location. A relevant IGP system is disclosedin U.S. Pat. No. 6,584,339 B2 (Galloway, Jr. et al.), the contents ofwhich is incorporated by reference herein. In IGP, physical space dataprovides three-dimensional (3D) coordinates for each of the physicalsurface points. Based on the physical space data collected, point-basedregistrations used to indicate position in both image space and physicalspace are determined. The registrations are used to map into imagespace, image data describing the physical space of an instrument used toperform the IGP, an anatomical region of interest and a particularportion to be studied (e.g., a tumor or growth). The image data isupdated on a periodic basis.

Further, preferred embodiments of the present invention utilize anoptically tracked two dimensional (2D) ultrasound probe to acquire a 2Dimage(s). Additionally, preferred embodiments of the present inventionmay also utilize an optically tracked 3D ultrasound to acquire 2D or 3Dimage(s). Embodiments of the present invention therefore permit thecreation of a 3D ultrasound volume from 2D tracked ultrasound data andcreation of a 3D ultrasound volume from 3D tracked ultrasound data.Acquired imaging scans (e.g., Computed Tomography (CT) scans) can beused as a comparison and/or in conjunction with pre-operative andinter-operative 3D ultrasound volume sets. The 3D ultrasound volume setsprovide the ability to be tracked over time.

Embodiments of the present invention include the creation of a 3-Dultrasound image volume set from tracked ultrasound data and allow forphysical space registration to patients. While described herein as usewith an endorectal ultrasound (ERUS) or tracked endorectal ultrasound(TERUS) probe 352 (FIG. 1), embodiments of the present invention are notlimited thereto. The concepts embraced by the embodiments of the presentinvention can be utilized with other ultrasound technologies and/or inother systems that provide “spatially-oriented” ultrasound images.Embodiments of the present invention may work in a range of ultrasoundapplications including echocardiography, cardiovascular ultrasounds,renal ultrasounds, obstetric ultrasounds, abdominal ultrasounds, breastultrasounds, gallbladder ultrasounds, transrectal ultrasounds,transvaginal ultrasounds or the like.

Various embodiments of the present invention include (i) a trainingsimulation tool using detailed spatially oriented images, (ii) aclinical tool using real time spatially oriented images with IGS; and(iii) methods of developing a spatially oriented image library(ies).

By creating an ultrasound volume set and accomplishing a registration tothe physical space of the patient, more diverse ultrasonography trainingcan be provided by using the same patient and changing image setsthereby creating a “virtual trainer” for ultrasound. The motion of theultrasound probe 352 is not generally as important to the trainingprocess, unlike virtual surgical training, as much as the imageinformation analysis and interpretation. Preferably, libraries ofultrasound images are created so that a student, such as an ultrasoundtechnician, radiology resident or a surgeon, can choose multiple sets ofdata to focus in certain areas. For example, libraries of data mayinclude pluralities of breast ultrasounds, gallbladder ultrasounds,transrectal ultrasounds, transvaginal ultrasounds, disease or traumaultrasounds or tumor ultrasounds in certain areas. The technology hasthe ability to lessen the cost of training, improve the accreditationprocess and remain applicable in all fields of ultrasonography.

Currently, various modules exist to acquire data and produce spatiallyoriented 3D images and volumes from a multitude tracked ultrasonicprobes 352. Volumes collected across a patient population can be used totrain ultrasound users who would otherwise not have access to such alarge set of images. For example, students could train using a kidneycancer volume set without having to scan a patient with kidney cancer.This allows for increased experience before a student actually performsan ultrasound exam clinically. A technician can learn the propertechniques to acquire valid images (see FIG. 13) that can be readilyinterpreted by a physician. Residents learn both the proper acquisitionand interpretation of ultrasound images. In some cases, particularlyobstetrics/gynecology (OB/GYN), nurses will acquire and assist ininterpretation of the images.

Besides ease of use, optical tracking has an advantage over the other 3Dacquisition methods which is a result of coordinate integrated imaging.When using a tracked ultrasonic device, such as a TERUS probe 352 (FIG.1), the exact location and orientation of the probe 352 are known.Through the process of calibration, the location and orientation of theultrasound beam are also known in an external coordinate space. Thisallows each pixel in the ultrasound data set to be assigned a 3Dcoordinate value in a physical space that is related to the ultrasoundspace through a specific transformation matrix. This method of assigningcoordinate values to the ultrasound data in physical space has twoadvantages. First, it allows the direct comparison of two imagingmodalities. This is achieved by transforming a data set, such as CT,into the same physical space as the ultrasound, making an accuratecomparison possible. Second this method allows localization of multipletools and image sets into the same physical space. The user then has theability to guide a tracked instrument, such as a biopsy needle orsurgical instrument, to a specific location in physical space while atthe same time viewing the progress in all imaging modalities (i.e.,ultrasound and CT). These co-registration and guidance techniques arenot possible using the mechanical 3D volume reconstruction methodsbecause multiple image sets and surgical tools cannot be localized inthe same physical space. The mechanical based methods are appropriatefor 3D volume reconstruction of ultrasound data, but are not valid foranything beyond visual enhancement of the rectum.

Referring to FIG. 1, an apparatus 300 that collects and processesphysical space data while performing an image-guided procedure on ananatomical area of interest includes a calibration probe 320 thatcollects physical space data by probing a plurality of physical points,a tracked ultrasonic probe 352 that outputs ultrasonic image data (2D or3D), a tracking device 325 that tracks the ultrasonic probe 352 in spaceand an image data processor 305 comprising a computer-readable medium(e.g., memory, FlashRAM, hard disk, etc.). The physical space dataprovides 3D coordinates for each of the physical points. Thecomputer-readable medium 305 holds computer-executable instructions thatinclude determining registrations used to indicate position in bothimage space and physical space based on the physical space datacollected by the calibration probe 320. The instructions further includeusing the registrations to map into image space image data describingthe physical space of the tracked ultrasonic probe 352 used to performthe image-guided procedure and the anatomical area of interest. Theinstructions also include constructing a 3D volume based on the 2D or 3Dultrasonic image data on a periodic basis.

In one preferred embodiment of the present invention described usingFIG. 1, an optically tracked endorectal ultrasound (TERUS) probe 352 isutilized for improving the care of rectal cancer. While described hereinas used with a TERUS probe 352, embodiments of the present invention arenot limited thereto. ERUS images are intrinsically different from imagestaken by CT or MRI in that ultrasound provides 2D images while CT andMRI provide 3D data sets that can be viewed as 2D images. By opticallytracking the TERUS probe 352, one may overcome the limitations of 2Dultrasound and improve the diagnosis and care of patients with rectalcancer. The TERUS probe 352 may be a 360-degree rotating BK 1850 TERUSprobe 352 (FIG. 2) commercially available from B-K Medical, Herlev,Denmark.

FIG. 1 shows that the ultrasound-based IGP system 300 includes anendorectal ultrasound probe 352 with an attached “gear shift knob” rigidbody 353 (shown in FIG. 2), an ultrasound machine 354, a referenceemitter 90, a calibration probe 320 and an optical tracking localizationsystem 340. The reference emitter 90 establishes an overall 3Dcoordinate system. The ultrasound machine 354 may be a BK Falcon 2101commercially available from B-K Medical (FIG. 5). The calibration probe320 (FIG. 4) may be a pen probe commercially available from NorthernDigital, Waterloo, Ontario, Canada. The optical tracking localizationsystem 340 may be an Optotrak 3020 commercially available from NorthernDigital.

The optical tracking system 340 determines triangulated position databased on emissions from a plurality of infrared emitting diodes (IREDs)distributed over the surface of a handle of the calibration probe 320,the TERUS probe 352 and/or another instrument. The optical trackingsystem 340 includes the optical tracking sensor 325 and optionally anoptical reference emitter 90. The optical tracking sensor tracks theIREDS that are disposed on the handle of the calibration probe 320 andIREDS disposed on the reference emitter 90. The reference emitter 90 isrigidly or semi-rigidly attached to the patient. FIGS. 1 and 4 show arectangularly shaped reference emitter 90, but other shaped referenceemitters 90 may be utilized such as a cross-shape reference emitter 90′(FIG. 7) and the like. The plurality of IREDs emit a plurality ofintermittent infrared signals used to triangulate the position of thecalibration probe 320 in 3-D image space. By using the point-basedregistrations and the triangulated position data to map into imagespace, image data describing the physical space of the distal end of theTERUS probe 352 can also be used to perform the IGP and to update theimage data on a periodic basis. Other image-tracking systems such asbinocular-camera systems may be utilized without departing from thepresent invention.

FIGS. 1 and 7 show that the optical tracking localization system 340includes a control box 315 to interface with a computer 305. Thesoftware program that allows the integration of the components may be anoperating room image-oriented navigation system (ORION), which wasdeveloped in the Surgical Navigation and Research Laboratory (SNARL) labat Vanderbilt University, Nashville, Tenn. ORION may be implemented inWindows NT using MS Visual C++ 6.0 with the Win32 API. ORION wasoriginally developed in Windows NT and is running on a 400 MHz processorpersonal computer (i.e., an image data processor) 305 with 256 MB ofmemory and a display monitor 310. However, other operating systems andprocessors may be utilized. The computer 305 may also include twospecialized cards such as a VigraVision-PCI card (commercially availablefrom VisiCom Inc., Burlington, Vt.) which is a combination color framegrabber and accelerated SVGA display controller which is capable ofdisplaying NTSC video images in real time, and an ISA high-speed serialport card communicates with the calibration probe 320 via the controlbox 315. Of course, the computer 305 may include the necessaryinterfaces and graphics drivers without the need from specialized cards.For example, optical tracking system 340 may include network connectionssuch as Ethernet, infrared (IR), wireless (Wi-Fi), or may include busadapters such as parallel, serial, universal serial bus (USB) and thelike.

FIG. 9 shows a basic software architecture for one possibleconfiguration of an image-guided tracking system 340 in accordance withpreferred embodiments of the present invention. The software may includedynamic link libraries (DLLs) such as localizer DLLs 405, registrationDLLs 410 and display DLLs 415. FIG. 10 shows a general flow chart for animage-guided tracking system for one possible configuration of animage-guided tracking system in accordance with preferred embodiments ofthe present invention.

Other hardware, operating systems, software packages, and image trackingsystems may utilized without departing from the present invention.

Locations of targets are found using two different imaging modalities,CT and ultrasound, and the target registration error (TRE) between thesetwo image sets are calculated. Fiducial markers 48 (FIG. 6) are used forimage registration. The fiducial markers 48 may either be skin markerssuch as those commercially available from Medtronics, Inc., Minneapolis,Minn., or bone implant markers such as those commercially available fromZKAT, Hollywood, Fla. The fiducial markers 48 are utilized to localizein the images using image processing routines and then touch using anoptical tracker in the operating room. The positions of the fiducialmarkers 48 are recorded and then a point registration is performed usingeither a quaternion based or singular-value-decomposition-basedalgorithm. Fiducial divot caps are used for finding the location of thefiducial markers 48 in physical space and fiducial CT caps are used forfinding the location of the fiducial markers 48 in CT space. Thesefiducial caps are interchangeable and an appropriate type is chosendepending on the desired imaging modality. Preferably, non-planarfiducial markers 48 are used to align the CT and Ultrasound images in arigid registration process.

For calibration, two rigid bodies are used that both have Infrared LightEmitting Diodes (IREDS) and are tracked by the optical trackinglocalization system. One rigid body 353 attached to the TERUS probe 352,and the TERUS probe 352 is securely fixed. The rotating tip of the TERUSprobe 352 is immersed in a bath of water (FIG. 3) or other suitablematerial. The ERUS mounted rigid body 352 functions as a reference towhich the second rigid body is tracked. The second rigid body is the penprobe 320 with a ball-tip such as a 3 mm ball-tip. The tip of thecalibration probe 320 is placed into the beam of the TERUS probe 352 andcan be seen as a bright spot in a corresponding ultrasound image (seeFIG. 11 with the location of the ball tip circled). Using theORION-based IGP system 300, which includes frame-grabbing capabilities,images along with the corresponding locations of the rigid bodies areacquired and saved. These images are processed to determine the“best-fit” plane through the points. The 2D location of the tip of thecalibration pen probe 320 is determined in each of the images. Aplurality of the 2D point locations are used to perform the calibration.The 2D locations are mapped to respective 3D locations using theLevenberg-Marquardt algorithm to solve for the resulting transformationmatrix and the pixel to millimeter scale factors in the x and ydirections. A subset of the acquired points or other acquired points canbe used as an internal check of the precision of the plane. The softwareprogram selects the points in a random order, so each time the programis run, there is a potential for a different solution because ofdifferent points used. The software program then reports the average andmaximum errors as well as the standard deviations for the calibrationpoints and the check points. The program also reports the scale factorsthat are used to map pixels to millimeters and the transformation matrixof the calibrated beam to the rigid body attached to the ERUStransducer. One important error to observe is the RMS error of the checkpoints (TRE). This is a quantification of plane fit quality. However, itis important to note that with one set of calibration data points, it ispossible to get a variation in the checkpoint TRE. This is because thedata points used to calculate the plane and the data points used tocheck the accuracy of the plane change each time the calibration is run.

It is generally desirable to recalibrate the TERUS probe 352 when therigid body 353 is removed and then reattached to the TERUS probe 352,and/or when the ultrasound is transmitted at a different frequency orviewed at a different field of view from the previous calibration. Thesecond condition is of interest to particular TERUS probe 352 describedabove because this TERUS probe 352 includes a multi-frequency transducerthat is easily interchangeable among the three frequencies and differentfields of view.

The next step is the registration of the CT volume to the physicalspace. This is accomplished by using the extrinsic fiducials 48 that areattached to the outside of the patient's body and using rigidregistration techniques. The (x,y,z) locations of the fiducials 48 arefound in the CT set and in physical space, and then registered using thequaternion method. To find the locations in CT space, athree-dimensional volume is created from the tomogram set. The fiducialcaps 48 used in the scan are radio-opaque and show up bright in theimages (see e.g., FIG. 14).

The centroid of each fiducial cap 48 is found using an intensity basedapproach and interpolation within slices to provide an accuracy of abouthalf of the slice thickness. This process is known as super-resolution.As previously described, the physical space locations are thendetermined using the tracked physical space calibration probe 320. Therigid registration is then performed with a subset of the fiducials 48from the two data sets. This registration creates a transformationmatrix that rotates and translates one set of data to match theorientation of the other set. This transformation matrix, along with theaverage error of all of the fiducials 48, are calculated. An accuratefiducial registration error is necessary for, but does not guarantee, anaccurate target registration error. Therefore, one fiducial 48 or anadditional fiducial 48 can be used as a target, like in the calibrationprocess, to check the accuracy of the FRE. The RMS error of theadditional fiducial 48 is reported as the registration TRE.

The final stage in the process involves finding the (x,y,z) locations ofthe targets in the CT tomograms the same way that the fiducial locationswere found. Then by using the output transformation matrix from theregistration process, the locations of those points in physical spaceare calculated. The locations of the targets are also found using theTERUS probe 352, and their respective (x,y,z) locations in physicalspace are calculated using the transformation matrices from thecalibration process and tracked image guided system. This provides twodata sets containing the locations of the same targets in physical spacelocated by two different methods. The values found using CT are taken tobe the actual locations, and the values found using the TERUS probe 352are compared to the actual. The distance between the locations of eachtarget is then found, and is recorded as the target registration error(TRE). It should be noted that an accurate TRE is the only trueverification of an accurately tracked system.

By optically tracking the TERUS probe 352 as data is collected, theintensity value for each pixel can be saved and then inserted into thenearest voxel in a corresponding volume matrix. Then validation of theaccuracy of a volume reconstruction can be performed by finding the 3Dcoordinates of targets that are inside of the volume and comparing themto known physical locations. Over-determining the data placed into thevolume around the area of interest is desirable so that speckle can bereduced and the signal to noise in the area will be improved byaveraging a particular pixel's value from different images.Transformation of a 2D coordinate location in ultrasound image spaceinto its corresponding 3D physical space occurs pixel by pixel and anyunfilled voxels are left empty. It is contemplated that vectormathematics can be utilized to speed up the process. During thetransformation process, the intensity value of each pixel is placed intothe appropriate 3D voxel. Multiple intensity values that map to the samevoxel in 3D space are handled. Preferably, arithmetic averaging is usedwhen multiple intensity values are mapped to the same voxel such thatthe average of the multiple intensity values is placed into the voxel.Alternately, when multiple intensity values are mapped to the same voxelin 3D space, the intensity value is overwritten into the voxel.

By moving the ERUS probe in and around a particular portion of theanatomy to be imaged and tracking the probe's motion, one obtains a setof data points that represents the ultrasound image at particular pointin the anatomy. Software is used to resolve the multiple data pointsinto a single volumetric data point (voxel) for each point in theanatomy of interest. That allows one to construct a 3D image of thatanatomy. The end result is a dataset consisting of ERUS voxels and theircoordinates in the 3D image. Thus, one has obtained a“spatially-oriented” ultrasound image of the particular anatomy ofinterest. This is analogous to having the coordinates for any point inthe volume, not just on the surface.

One can then manipulate (rotate, slice, pan, zoom, etc.) the data in anyfashion desired to reconstruct views of the anatomy of interest from anyperspective desired. This is analogous to the ability to manipulate CTand MRI images except the resolution of the internal detail is less thanwith either of these modalities; however, in colorectal ERUS is thecurrent standard of care for tumor diagnosis.

The incorporation of IGP techniques to pre-operative staging,inter-operative visualization and post-operative follow-up of rectalcancer using TERUS improves the accuracy of staging, reduces the overallmorbidity and mortality rates, and assists clinicians to detect andfollow recurrences of rectal cancer over time. Accordingly, compilingthe TERUS-obtained data into a library 100 (FIG. 11) of transrectalscans for interpretation. The library 100 may include scans of healthypatients, patients having tumors and, for patients having tumors,various stages of tumor growth.

Embodiments of the present invention create the 3D volumetric ultrasoundimages to create a library 100 (FIG. 11) of images (FIG. 14) that can beused for training via simulation. The library 100 and a means to displaythe data 310, such as a computer monitor 310 or a projector (not shown),can be used to provide interactive training of resident physicians,ultrasound technicians and nurses in effectively utilizing ultrasoundtechnology.

Referring to FIG. 11, the library 100 is created from 3D ultrasoundvolumes taken of multiple patients or multiple anatomies. Theultrasound-based IGP system 300 can acquire the images to be used in thelibrary 100 and can be used for the training phase as well (FIG. 12).Preferably, the library 100 is developed by using the system 300clinically. The more the ultrasound-based IGP system 300 is used, thelarger the library 100 will become. The library 100 can be grouped basedon patient characteristics, e.g., anatomy, disease, age, gender or thelike. The specific volume sets can then be included back into the system300.

It is contemplated that the library 100 is developed on a central server(not shown) by uploading data from a plurality of ultrasound-based IGPsystems 300. The plurality of ultrasound-based IGP systems 300 can benetworked or simply dial-up. The plurality of ultrasound-based IGPsystems 300 may communicate through the internet. Alternatively, thelibrary 100 may be developed by separate submissions to a common sourcefor compilation onto a distributable storage medium such as a DVD,CD-ROM, Optical Disk or the like.

Although the ERUS technique was described above, the concept is readilyadapted for training purposes for imaging other parts of the anatomy byusing different ultrasound probes 352. Cardiac, OB/GYN, abdominal andcarotid imaging all make wide use of ultrasound imaging.

The ultrasound-based IGP system 300 stores a library 100 of“spatially-oriented” images for various portions of the anatomy. Astudent practices the use of ultrasonography on a person such as hisinstructor or a classmate, but the images that are displayed would bethose in the library 100. For example, suppose the instructor wanted todemonstrate and have the student practice the features of acquiring andunderstanding colorectal tumors. Referring to FIG. 12, the studentactually manipulates the tracked ultrasound probe 352 or simulatesmotion of the tracked ultrasound probe 352 in use. The computer display310 presents images of a tumor or an area of interest stored in thelibrary 100 from many different geometric perspectives to facilitateunderstanding by the student. In this fashion it is envisioned that thetraining interval on real patients could be shortened by reducing (butnot eliminating) the number of live patients on which the student mustbe trained.

The library 100 of spatially-oriented images stores a plurality ofdifferent patient datasets for training and comparison. One of theadvantages of this system is that it can acquire the images to be usedin the library and can be used for the training phase as well. FIG. 11shows a general flow chart for developing the training and diagnosticslibrary in accordance with preferred embodiments of the presentinvention. FIG. 12 shows a general flow chart for training using thetraining and diagnostics library.

Currently, modules exist to acquire data and produce spatially-oriented3D images of the colorectal anatomy. The software for such is verymodular and modules for cardiac, carotid, etc. can be readily added.

Of course, the preferred embodiments of the present invention are notlimited to endorectal imaging and evaluation and may be utilized toanalyze other anatomical regions of interest such as the vagina, theuterus, colon, upper intestine, throat and the like. The preferredembodiments may be utilized in conjunction with laparoscopic andendoscopic surgical techniques, as well, such as by inserting anultrasonic probe 352 into a body through a cannula.

While described above as being used in combination with a CT scan, otherimaging techniques such as PET, MRI or the like, may be utilized aloneor in combination.

The ultrasound-based IGP system 300, when used with a second imagingtechnique (e.g., CT, PET, MRI or the like), enables other analyses suchas size changes of a target (e.g., a tumor) in an anatomical region ofinterest (e.g., the rectum). Therefore, training using the combinedimaging modalities stored in the library is also desirable.

The ultrasound-based IGP system 300 can also be used as a clinical toolfor diagnosis of a disease. In this case a volume around a region ofinterest would be created and then sent to a clinical expert forevaluation. The clinical expert then reports the diagnosis. Thisinformation is sent back to the person who collected the data set and isalso stored along with the image volume in the library.

The ultrasound-based IGP system 300 can be used as a clinical tool forthe treatment of a disease. An ultrasound volume is created and loadedinto the same or another ultrasound-based IGP system 300 for use duringa procedure in the clinic or operating room (OR).

The present invention may be implemented with any combination ofhardware and software. If implemented as a computer-implementedapparatus, the present invention is implemented using means forperforming all of the steps and functions described above.

The present invention can be included in an article of manufacture(e.g., one or more computer program products) having, for instance,computer useable media. The media has embodied therein, for instance,computer readable program code means for providing and facilitating themechanisms of the present invention. The article of manufacture can beincluded as part of a computer system or sold separately.

From the foregoing, it can be seen that the present invention comprisesa method and apparatus for standardizing ultrasonography training usingimage to physical space registration of tomographic volumes from trackedultrasound. It will be appreciated by those skilled in the art thatchanges could be made to the embodiments described above withoutdeparting from the broad inventive concept thereof. It is understood,therefore, that this invention is not limited to the particularembodiments disclosed, but it is intended to cover modifications withinthe spirit and scope of the present invention as defined by the appendedclaims.

1. An apparatus that collects and processes physical space data whileperforming an image-guided procedure on an anatomical area of interest,the apparatus comprising: (a) a calibration system that collectsphysical space data, the physical space data providing three-dimensional(3D) coordinates; (b) a tracked ultrasonic probe that outputs ultrasonicimage data; (c) a tracking device that tracks the ultrasonic probe inspace; (d) an image data processor comprising a non-transitorycomputer-readable medium holding computer-executable instructionsincluding: (i) based on the physical space data collected by thecalibration system, determining registrations used to indicate positionin both image space and physical space; (ii) using the registrations tomap into image space, image data describing the physical space of thetracked ultrasonic probe used to perform the image-guided procedure andthe anatomical area of interest; and (iii) constructing a 3D volumebased on the ultrasonic image data on a periodic basis; and (e) astorage medium that stores a plurality of 3D volumes acquired by theimage data processor.
 2. The apparatus according to claim 1, furthercomprising: (f) a scanning device for scanning the respective anatomicalarea of interest of a patient to acquire, store and process a 3Dreference of tissue, wherein the image data processor creates a scannedimage based on the scanned tissue, the storage medium also stores aplurality of scans associated with the plurality of 3D volumes.
 3. Theapparatus according to claim 2, wherein the scanning device is one ofcomputed tomography (CT), magnetic resonance (MRI), PET and SPECT(single photon emission computed tomography).
 4. The apparatus accordingto claim 1, wherein the computer-executable instructions furtherinclude: stored image/volume retrieval and display instructions in orderto selectively retrieve and display one of the plurality of stored 3Dvolumes.
 5. An apparatus that collects and processes physical space datawhile performing an image-guided procedure on an anatomical area ofinterest, the apparatus comprising: (a) a calibration system thatcollects physical space data, the physical space data providingthree-dimensional (3D) coordinates; (b) a tracked ultrasonic probe thatoutputs ultrasonic image data; (c) a tracking device that tracks theultrasonic probe in space; (d) an image data processor comprising anon-transitory computer-readable medium holding computer-executableinstructions including: (i) based on the physical space data collectedby the calibration system, determining registrations used to indicateposition in both image space and physical space; (ii) using theregistrations to map into image space, image data describing thephysical space of the tracked ultrasonic probe; and (iii) constructing a3D volume based on the ultrasonic image data on a periodic basis; and(e) a storage medium that stores a plurality of 3D volumes acquired bythe image data processor, the computer-executable instructions includingstored image/volume retrieval and display instructions in order toselectively retrieve and display one of the plurality of stored 3Dvolumes.